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1.1 Introduction 


In recent years, both multilingualism and multimodality have become 
vibrant and urgent fields of linguistic investigation. However, while the 
phenomena occur extensively in both modern and historical contexts, 
and frequently in the same texts, there is a significant lack of scholarship 
that would address the many links between them. Multimodal context- 
ualisation cues evoking communication beyond a given text or utterance 
are tightly fused with language-based processes of meaning-making. In 
historical texts and contemporary social media communication alike, 
pre-attentive engagement techniques (Bateman 2008; see also Peikola and 
Varila, Chapter 5, this volume) are implemented by content producers 
and guide processing. Orthographic aesthetics encodes social evaluations 
and commentary of (linguistic) otherness in the 19th century (Tyrkk6 and 
Legutko, Chapter 7, this volume), while visual diamorphs, brevigraphs 
and non-alphabetic symbols embody the prestige of a medieval deluxe 
manuscript (Rogos-Hebda, Chapter 2, this volume). The voices of others, 
discourse and writing representation, and perspective shifts in gen- 
eral frequently receive graphic and typographic marking in mono- and 
multilingual texts alike (e.g. Claridge 2020; Grund and Walker 2020). 
Indeed, bilingualism is as purposeful and commercially devised a tool as 
the graphics and photos placed in blogs and vlogs by micro-influencers 
(Limatius, Chapter 9, this volume). It is thus important to redress the 
interfaces of multilingualism and multimodality by paying attention to 
features such as graphics, layout, boundary marking or typography as 
essential and inseparable from the verbal content. 

Although recent technological advances have perhaps intensified inte- 
gration of the multimodal and the multilingual, the dynamic relationship 
between the two pre-exists modern digital media. In order to understand 
the interplay between the two phenomena, in-depth scholarly explor- 
ation is needed in order to discover the dynamics and trends that have 
endured and developed over centuries, as well as those that were spe- 
cific to particular circumstances. The premise of this volume is that the 
interfaces between multiple codes and modes (i.e. multilingualism and 
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multimodality) need to be explained by adopting a systematic approach 
that takes a long historical view of and a rigorously theoretical, but 
evidence-based, perspective to the phenomena. 

This volume contributes to currently ongoing efforts to find semiotic 
frameworks that afford more attention to complex literacy practices and 
multimodal contextualisation cues (e.g. Sebba 2012: 2; cf. Barret et al. 
2016; Peikola et al. 2021) as important windows into the interaction of 
two or more linguistic codes. While not detaching multilingual writing 
from its social, pragmatic and individual dimensions, such approaches 
no longer privilege decontextualised linguistic data as central, but rather 
posit the need to pay equal attention to the different modes through 
which multilingual texts communicate. Earlier studies employing notions 
of transmodality and translingualism (e.g. Pennycook 2004) in reference 
to the broader semiotic theories, such as the multimodality proposed by 
Kress and van Leeuwen (1996, 2001) or geosemiotics (Scollon and Scollon 
2003), have in recent years paved the way for new understandings of 
multilingualism as “a complex of specific semiotic resources” (Blommaert 
2010: 102). First and foremost, these frameworks respond to the challenges 
of technologisation and globalisation of communication observed over 
the last three decades, particularly in the digital sphere (Leppänen et al. 
2017), and we will argue that the same frameworks afford important 
insights into earlier time periods as well. Still, the interfaces between 
multilingual practices (Pahta et al. 2017), visual pragmatics (Carrol et al. 
2013; Peikola et al. 2017) and other modality features remain woefully 
under-theorised (e.g. Kopaczyk 2017). The chapters brought together in 
the volume use data from a broad array of contexts, periods and genres, 
posing the following general questions: What kinds of interfaces between 
mode(s) and code(s) can research identify? What are the extralinguistic 
modes evoked by specific (groups of) features? To what extent do these 
organise the text? Does the principle of resource integration, i.e. the view 
that modes are holistic units (Norris 2004; cf. multimodal ensembles in 
Kress 2010) apply to text-based multilingual communication? What can 
we gain by appreciating that the interfaces between code(s) and mode(s) 
are meaningful (semiotically charged)? Providing tentative answers to 
these questions constitutes the contribution of studies conducted primarily 
within the frameworks of linguistics to the field of multimodality studies. 
By exploring the interface of multiple codes and modes, the chapters we 
have collected afford new insights that cannot be gained from studying 
multilingualism or multimodality alone. 

Pursuing our unifying focus, we need to reconcile an internal contradic- 
tion between the nature of the data under analysis: namely the apparently 
monomodal multilingual written text, which belies a much more com- 
plex semiotic sphere, and multimodality, both as a phenomenon and as a 
research framework. To a linguist, conventionally, a text is a domain of 
the word, and historical written data even more so. For a multimodal per- 
spective, on the other hand, image, gesture, speech, even human presence 
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in communication are required to start with. Nevertheless, any attempt to 
rebalance the analytical focus of linguistic phenomena by including long- 
neglected aspects of multimodality must be done carefully and without 
losing sight of language use. 

Multimodality does not necessarily need to lie beyond language, as 
this volume sets out to show. On the contrary, language(s) may be used 
as (a) powerful lense(s) through which “the multitude of ways we com- 
municate” (The Routledge Handbook of Multimodal Analysis, Jewitt 
2017: blurb) may be recovered for the past and present. In this volume, 
multilingual practices (in the sense defined by Pahta et al. 2017) in par- 
ticular are studied with a common purpose of looking through rather 
than beyond language to integrate the non-verbal aspects of communica- 
tion into a linguist’s perspective. 


1.1.1 Terminological Preliminaries and Limitations 


Current research tends to view multimodality “as a resource for com- 
municative expression” (Bateman 2014: 25), while much of the research 
into multilingualism in the globalised world has argued that using more 
than one tongue involves “a complex of specific semiotic resources” 
(Blommaert 2010: 102). These general statements, apt and concise as 
they are, apparently lead in the same direction by indicating reservoirs 
of knowledge, skills and forms of expression that are open to selection. 
However, there may be different ways of interpreting the use of such 
reservoirs. In the past, multimodality may have lain in underlying oral 
practice or in the intended performative nature of the original text, which 
only survives today in written format. In the present, multimodal scope 
involves a range of visual and interactive tools of communication not 
only between humans, but also increasingly with the participation of arti- 
ficial intelligence (AI). Likewise, multilingualism may equally be seen as 
a professional/social asset or as a disadvantage, while its manifestations 
and scope are diverse and dynamic. 

In line with recent research into multimodality, we accept that it 
involves “activation of extra-linguistic knowledge and metaphorical 
associations” (Lyons 2018: 21) and consequently we opt to study it in 
written/text-based contexts for reasons elucidated above. In other words, 
multimodality does not only depend on the presence of an extralinguistic 
element (e.g. image) that is an integral part of a text encoded in its pro- 
duction, but also in the processes of its decoding, that is, multimodality is 
discursively evoked (Lyons 2018). Indeed, we want to make the case that 
processes that constitute multimodality occur parallel to the activation 
of cognitive capacities enabling the use of more than one language and 
the contextualisation of meaning-making in the relevant repositories of 
communal and individual experiences. Although access to the interfaces 
of the two “activations” will never be complete, we believe that even 
an atomistic approach whereby specific features of code interacting with 
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mode are teased out will bring us closer to understanding the complexity 
of the processes involved. 

Below, we provide a clarification of some key notions which we con- 
sider fundamental to our interest of interfacing between the two fields, 
i.e. mode and code respectively, in an attempt to sketch a preliminary 
background for the chapters presented in the volume. 

In her studies into medieval manuscript miscellanies as not only gen- 
erically, but also linguistically heterogenous material (2015, 2016), 
Kate Maxwell levels criticism at multimodality studies: “[i]n terms of 
multimodality as a discipline, the basic process of identifying semi- 
otic modes is one which leaves aside aspects of a culture which are of 
supreme importance for an understanding of that culture” (2015: 359). 
In order to overcome this limitation, i.e. to incorporate culture, or rather 
build a bridge that allows insights into the past in this respect, Maxwell 
conceptualises mode as a three-layered concentric construct. The outer 
layer encompasses cultural practices (e.g. reading and writing), the 
middle one covers semiotic resources (for medieval manuscripts (MSS), 
e.g. ordering and page layout), and the inner layer houses the so-called 
elements (Latin pars, e.g. scripts or punctuation). The model thus frames 
the notion of mode into a general context of culture. Framing, however, 
is only illusive here. 

Despite the concentric conceptualisation, ordering is actually decep- 
tive while the layers are not straightforwardly hierarchical or imperme- 
able: i.e. a specific feature (e.g. word spacing, layout) may potentially 
be an element, a semiotic resource or a cultural practice (2015: 360). 
Maxwell indicates language as one such feature. Our contributors have 
focused primarily on the choices and boundaries between languages/ 
codes and have attempted to recover their relationships to a/the relevant 
level(s) in the three-dimensional model of mode. In doing so, the chapters 
in the volume present a nuanced mapping of correspondences between 
language(s), elements, semiotic resources and cultural practices, revealing 
both changes as well as stable patterns over time. This is only possible 
through an array of texts witnessing cultural practices at different points 
in time, different cultural circles and representing different discourse 
domains. As the interface between multiple modes and codes over time is 
a largely unexplored field of research, the nature of the chapters included 
in the volume is to some extent exploratory. In the future, some of the 
data, periods and practices brought together here will, hopefully, give rise 
to more fine-tuned emphasis on those cultural practices (as constituents 
of mode and modality) whose meaningful interfaces with multilingualism 
prove most robust theoretically and methodologically. 


1.1.2 Volume Structure 


Some of the chapters in the volume focus on texts characterised by visible 
non-verbal cues relevant to their multilingual nature. We propose that 
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in these texts semiotic resources and elements are the site of mode(s). 
This means that boundaries and interactions of the two or more codes 
materialise on the page, thus evoking extralinguistic associations through 
visual features and/or spatial organisation. In other texts, the mode vs 
code interface is external to the text as an artefact, meaning that it is 
not revealed on the page and extends to the processes of text transmis- 
sion. Thus it lies in the social materiality of meaning-making, which in 
Maxwell’s model (2015) may conveniently be put under the label of cul- 
tural practices. 

In reflection of the two main foci of the studies, the volume has been 
divided into two parts: (1) Multilingualism vs semiotic resources and 
elements, and (2) Multilingualism vs modes as cultural practice. In Part 1, 
the features analysed in the context of multilingual practices include: type 
and size of script, ink colour, mise en page, scribal abbreviations (including 
non-alphabetic symbols, e.g. brevigraphs, Rogos-Hebda, Chapter 2); 
code boundaries manifested in layout and spacing (Seiler, Chapter 3; 
Włodarczyk and Adamczyk, Chapter 4), graphics and tabulations (Peikola 
and Varila, Chapter 5). In a theory-oriented mode, the paper closing 
Part 1 summarises the inventory of cues to codes and modes by pro- 
posing a hierarchical and transhistorical model that operationalises the 
relationship between multilingualism and multimodality (Kopaczyk, 
Chapter 6). 

In Part 2, cultural practices entailed in multilingual texts and practices 
are viewed in the context of social media networking and interaction 
as well as in the world of product packaging. The paper by Tyrkk6 and 
Legutko (Chapter 7) explores the social materiality of meaning-making 
by indicating evaluation of linguistically foreign and non-standard 
voices encoded in orthography and image. Othering and social com- 
mentary expressed on the pages of the 19th-century popular magazine 
are paralleled in present-day multimodal communication through social 
media and surface as identity statements in reference to gender (Radke 
and Versloot, Chapter 8), self-branding in influencer culture (Limatius, 
Chapter 9) and the ordering of languages in multilingual product pack- 
aging (Sebba, Chapter 10). 

Illuminated medieval manuscripts provide fruitful ground for studying 
the interface of multilingualism, semiotic resources and elements as 
Rogos-Hebda shows in Chapter 2. In medieval times, manuscript- 
based textuality relied on a combination of different semiotic resources, 
while cognitive consumption of such artefacts was as much visual as 
spatial-tactile experience, not unlike the multisensory experience of 
present-day electronic communication. The chapter focuses on the multi- 
modal contexts for visual code-switching in two 15th-century deluxe 
manuscripts of Gower’s Confessio Amantis and investigates “visual 
practices” of two distinct scribal communities responsible for copies of 
the same Middle English text. These practices are viewed as pragmatic 
markers in the perspective of the pragmatics on the page (Carroll et al. 
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2013). Inherent multilingualism of institutions and literary endeavours 
fused Latin and Anglo-French with the vernacular, but in later Middle 
English, unlike directly after the Norman Conquest, the different codes 
were highlighted by means of colours. 

The analysis of the flagging of language switches is framed within 
Sebba’s visual code-switching theory, which distinguishes between 
discourse-related (titles, headings, paragraphs) and visual/spatial 
units (frames, boxes, columns or delineated borders), among others. 
The former ensure structuring and cohesion, whereas the latter delin- 
eate physical space for code-switching (Sebba 2012: 106). The analysis 
indicates significant differences between the two MSS, but shows for both 
that Latin units have clear indexing, rather than just an authoritative 
function. In addition, they may also be seen to play social/interactional 
pragmatic roles compared to the English text which is more content- 
oriented (cf. the triad of paratext, metadiscourse and framing, Bos and 
Peikola 2020: 12). Moreover, in MS R.3.2, language neutral pragmatic 
markers, such as paraphs, are shown to control reader engagement by 
accompanying the shifts to Latin and thus separating the multilingual 
practice from the English matrix. Latin may also express authorial stance, 
as the abbreviations utilised in the Latin fragments provide a commentary 
on the relative prestige of the copy, the work and the author himself (cf. 
Pearsall 1989 on auctoritas). Paradoxically, as the chapter demonstrates, 
the visual-spatial form of Latin units renders them both external to 
the primarily English text and, through their pragmatic and discourse- 
organisational significance, a necessary part of it. 

Chapter 3, Seiler’s contribution, presents a multimodal analysis of 
multilingualism in an intrinsically multilingual genre, a glossary. The 
chapter focuses on six medieval English glossaries, dating from the late 
7th to the middle of the 13th century and relies on the assumption that the 
layout of medieval glossaries represents an important semiotic resource 
employed to facilitate access, identify languages or establish a semantic- 
syntactic relationship between multilingual word pairs. Besides, patterns 
of layout organisation afford some insights into specific cultural practices 
and multimodal literacy. Seiler implements Sebba’s (2012) framework 
and focuses on language-spatial, language-content relationships and 
language-mixing types. The language-spatial relationships are found 
to be typically symmetrical as their content in the different languages 
is equivalent or overlapping. In terms of language mixing, types form 
diverse patterns ranging from contrasting monolingual columns, or rows, 
to mixed-language text block. Interestingly, despite the considerable time 
span of data, languages in medieval English glossaries are clearly ordered 
with Latin placed in the privileged slots on the left side of the entry or 
on the base line. In the trilingual glossaries, however, Anglo-Norman 
precedes Middle English in the two herbals, but follows it in the Stowe 
glossary. The chapter shows that visual/spatial features of glossaries may 
be design universals based on pervasive cultural practices (e.g. listing, 
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alphabetisation, reading direction from left to right). Nevertheless, the 
placement of interpretamenta above the corresponding headword is a spe- 
cifically medieval feature. This arrangement exploits a template provided 
by interlinear glossing as a cultural practice, which codifies the position 
above a word as an explanation slot. 

Boundary marking, another aspect of page layout in multilingual 
texts and its cross-modal nature (van Leeuwen 2004: 15) are the topic of 
Chapter 4 which analyses (meta)linguistic and visual features (code and 
discourse transitions) in the late 14th- and early 15th-century land books 
from Greater Poland, written in Latin with a consistent vernacular com- 
ponent, i.e. the Old Polish witness oath. Włodarczyk and Adamczyk view 
layout organisation and boundary marking in terms of global (on the 
page level) and local (on the grapheme level) visibility and suggest that 
the patterns of page organisation form a continuum with respect to global 
visibility, ranging from visually salient types (boundaries and transitions 
clearly marked and page layout is transparent), to those which show no 
orderly organisation and no visual marking of discourse boundaries. The 
chapter proposes a typology of the page layout patterns with respect to 
integration/separateness of the discourse/code components of the record. 
Four major types of boundary marking that capture transparency of page 
layout and the fluctuating salience of the mediating items can be observed 
in the records. This formal variation, as the chapter shows, also involves 
a diachronic and diatopic dimension. Diachronically, for instance, a 
(significant) increase in the number of rotas representing Type 1 (salient 
boundary marking and transparent page layout) over four periods of time 
is established, while an opposite tendency for Type 3 (no clear delimita- 
tion of the individual components of the record) is also observed. These 
results may be viewed as growing consistency over time in terms of a 
preference for a transparent structure, with the Latin and Polish parts 
of the record visibly separated. This pattern may have served the pur- 
pose of reader guidance. In terms of multimodality, the analysis confirms 
earlier research showing that cues realised in different modalities tend to 
reinforce one another (e.g. Fenlon 2015). 

Access structures encoded in multimodal features are explored in 
Chapter 5 based on a different contemporaneous genre: the late medi- 
eval calendars (1300-1550). They were popular Latin and/or vernacular 
tabularised presentations of names of saints and major festive days, but 
also included liturgical, astrological and medical information. Peikola and 
Varila view this largely unexplored material in the framework of genre 
studies underlining multiple literacy competencies (linguistic, numeric 
and symbolic) required to read the heterogenous data and to process 
the condensed information they recorded. A multimodal take proposed 
in the chapter singles out spatial features and layout as indicators of 
readers’ engagement at the level below awareness (Bateman’s 2008: 115 
pre-attentive engagement). This, the authors assume, means that visual 
access structures played a crucial part in the consumption of these texts. 
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As far as multilingual features are concerned, like other contemporary 
documentary texts, calendars may have been undergoing the process of 
vernacularisation in the period under scrutiny and exhibit interesting 
multilingual patterns. 

The qualitative analysis of 31 calendars in which Latin and the ver- 
nacular co-occur has been conducted on four levels of manuscript con- 
text: macro-level, calendar page (usually one month), calendar table, 
micro-level (core column, month of July). Three types of constraints 
are of relevance (Bateman 2008: 17-18), pertaining to canvas, produc- 
tion and consumption. The study established that on the macro-level, 
calendar design was determined by the nature of manuscript (religious 
vs astro-medical) and rendered prototypes of distinct subgenres, though 
hybrid material was also observed. At the micro-level, the visually salient 
KŁ abbreviation and the consistently Latin hexameter about the Egyptian 
days were typically placed in the upper region of the calendar page, which 
is understandable in the face of present-day-eye-tracking studies (upper 
left region fixation of the gaze on a page). In terms of multilingual elem- 
ents, later calendars showed a more frequent occurrence of these, which 
the authors see as symptomatic of vernacularisation. Typically for medi- 
eval and later utilitarian texts, calendars used visual diamorphs, i.e. non- 
language-specific abbreviations, which may be viewed as genre markers 
and access structures at the same time. 

Interfaces of multilingualism and multimodality remain under- 
theorised and this gap is addressed in Chapter 6, devoted to the structural 
model of multilingual practice (SMMP) proposed by Joanna Kopaczyk. 
This descriptive model is based on specialised multilingual historical 
texts and it attempts to conceptualise and operationalise the relationship 
between multilingualism and multimodality, in particular in terms of the 
visual cues vs linguistic content. Visual marking vs the lack of it at the 
point at which changes in linguistic resources take place (i.e. switches 
from one language or code to another) are the parameters of the model. 
A conventional structural linguistic hierarchy is extended to cover the dis- 
course and macro-genre levels and provides the backbone of the hierarch- 
ical organisation of the model. The SMMP is a transhistorical extension 
of an earlier historical model proposed by Kopaczyk. The applicability 
of the model to multilingual contexts is illustrated with an example of a 
sample of Scots laws from early modern Scotland where a complex inter- 
play between Latin, Scots and English is observed. 

Apart from breaking separable languages down to their structural 
ingredients, the model underlines the need for asking a range of questions 
related to the multilingual repertoires and user choices with respect to the 
marking in visual modalities. The systematic approach proposed in the 
model involves some sensitive areas that require further elaboration: bound- 
aries and shifts between languages and linguistic levels and the visual cues. 
Illustrations from historical administrative texts in Latin, English and Scots 
indicate the major points of complexity. Although SMMP has not so far 


Modalities from Medieval to Modern Times 9 


been applied systematically to a larger data sample, it is easy to imagine 
its practical implementation, in particular in the realm of corpus design. 
So far a consistent recognition of multilingual elements has rarely been the 
basis for the creation of historical or contemporary corpora. Multimodal 
features have been much more neglected due to the difficulty and modal 
heterogeneity of relevant tagging systems. Thus, the conceptualisation 
developed in the chapter might come in useful in tackling the challenge 
of creating multimodal and multilingual corpora that aim at “identifying 
different languages in written texts and relating the changes in the linguistic 
repertoire to non-linguistic meaning-making practices”. 

Opening Part 2 of the volume, Chapter 7, by Tyrkk6 and Legutko 
takes us to the Late Modern period. It was then that growing literacies, 
new genres and changing functions of communication in the public 
realm complexified interfaces between the mode and code even further. 
The boom of popular press transforms access structures in public and 
opinion-shaping texts. The culturally iconic periodical Punch published 
in London in the 19th and 20th centuries and read mostly by the 
growing bourgeousie and gentry provides visual and textual data for the 
chapter. Non-standard and foreign voices occurring in the 550 issues of 
the magazine are set in the framework of orthographic aesthetics (Jaffe 
2000: 509-511). The authors view such textual elements as reinforcing 
otherness through a disruption of the “normal meaning-making pro- 
cess” and reveal the social significance of orthography. This significance 
may be understood in terms of authorial evaluation of standard vs non- 
standard as previous research has shown (Casas 2009). In fact, marked 
representations of the voices of others have been interpreted before in 
terms of Goffmann’s changing hats (Goffmann 1959) as well, but Tyrkkö 
and Legutko provide a focused model and entrench orthographic features 
within a multi-code and multi-mode context. Apart from orthography 
and typography, the multimodal focus of the chapter lies in the interplay 
of text and image presented in cartoons and within cartoons (captions). 

Code switching was a common feature of Punch for the better part 
of its history, i.e. from 1840 to 1920, though the repertoire of distinct 
languages was rather limited. French segments tended to be marked 
typographically with some consistency, Latin was much less frequent 
and typical for academic contexts. As for switches into dialect, their aim 
was reader entertainment rather than derision of the presented speakers. 
Typographic highlighting and orthographic norm-deviating spellings 
achieved a range of social and presentational goals, primarily that of witty 
commentary mixed with the need for maintaining hierarchies and norms. 

Chapter 8 by Radke and Versloot reaches for a framework of referen- 
tial modality which enables comparing and contrasting different modal- 
ities of distant and direct communication in terms of the performance 
of gender and identity. The German-Namibian diaspora in Germany 
provides a well-specified context for the study of multilingual and multi- 
modal practices in computer-mediated communication (CMC) and FTF 
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communication. Cultural contextualisation of the data follows criticism 
of traditional multimodality studies which tend to overlook this issue 
(Maxwell 2015: 359). Data has been divided into mixed (CMC and 
FTF) and single mode samples (CMC). Using the framework of refer- 
ential modality and methods of discourse analysis, Radke and Versloot 
observe and quantify some differences in the performance of gender in 
terms of the use of English-derived and Afrikaans-derived keywords 
divided into function and content words and the occurrence of multi- 
lingual brocatives. Referential modality is understood as “the use of one 
or more claims referring to a mode different from the mode in which the 
claims occur”, i.e. in particular direct and indirect quotes, and it shows 
that mixed mode environments are more conducive to the fostering and 
strengthening of social bonds among members. 

Some approaches to code-switching indicate a connection between the 
use of non-standard language, frequency of language and gender showing 
that men rather than women switch languages more often because of 
their general orientation towards covert prestige. The results of the ana- 
lysis of communication within a German-Namibian diaspora contradict 
this connection as they show that women tend to use more non-German 
lexical items than men. Second, a discursive analysis reveals that informal 
language features, such as the use of brocatives, characterises male com- 
munication patterns and indexes in-group cohesion and cameraderie. 
Digital gender roles are thus visible both in the use of multilingual items 
(more common in women) and informal multilingual features such as 
brocatives (more frequently used by men). Digital gender roles, as the 
chapter shows, cannot be viewed as established, but are clearly performed 
and shaped differently by single- and mixed-mode environments, while 
the framework of referential modality enables incorporating the multi- 
lingual nature of communication in the context of this specific diaspora. 

Digital gender roles are also clearly present in influencer culture, a 
prominent aspect of social media, which is the theme of Hanna Limatius’s 
Chapter 9. Here, the focus falls on a gender profiled web community of 
fashion and beauty microcelebrities and their “followers”. This qualita- 
tive study focuses on interfaces of visual and linguistic representations 
viewed as discursive, aesthetic and social resources in samples of blogs 
and vlogs (YouTube videos and Instagram posts). Influencer culture is pri- 
marily a commercial phenomenon and the study opens a line of research 
that foregrounds the benefits that both multimodal and multilingual 
practices may have for influencers. The major assumptions behind the 
study are that multimodality enriches interaction with cues not present 
in text-based online communication, while multilingualism is mostly a 
means of identification and self-expression on social media. If combined, 
the two might be fostering authenticity, which may be one factor behind 
the commercial success even of microcelebrities. 

The analysis focuses on aesthetic and social aspects of blogs, the mutual 
influence between visual and verbal content, and interactive features, i.e. 


Modalities from Medieval to Modern Times 11 


commenting, sharing and embedding (of YouTube content, for instance). 
On the one hand, blogs emerge as being conceptualised in terms of a 
brand or product (branding and tagging), while at the same time providing 
space for conversation and exchange and community-building. Visual 
resources employed by microcelebrities (props, colours and framing) 
are consistent with both concepts behind individual blogs and intended 
audience. In addition, results of the analysis supported by metatextual 
comments show that the multilingual practices of Finnish influencers 
on YouTube, i.e. switches to English, may be spontaneous rather than 
a strategy of reaching an international audience. On Instagram, how- 
ever, Limatius notices purposeful bilingualism, with Finnish translations 
of English captions, which implies that the audiences on the platforms are 
not the same and that influencers consult statistical data, e.g. location of 
their viewers (engaged audience) and adjust their practices accordingly. 

The final chapter (Chapter 10) by Mark Sebba proposes a prelim- 
inary theoretical approach to the analysis of multilingual product 
packaging. Product packaging is viewed as part of the semiotic land- 
scape and as language-defined objects. A collection of 80 items ori- 
ginating from 16 different countries (UK, Austria, Belgium, Canada, 
China, France, Germany, Italy, Japan, Macedonia, Netherlands, Poland, 
South Africa, Switzerland, Turkey, USSR) is approached from two 
perspectives: multimodality, where texts function both as language and as 
images, with semiotic interactions between the two; and multidimension- 
ality, where product packages are analysed as three-dimensional objects 
(in contrast with two-dimensional signage, for example). For this reason, 
information contained in packages needs to be spatially structured in 
three dimensions which may represent informational or sociolinguistic 
hierarchies of different languages. The analysis is based on a typology of 
messages drawn from Thomas (2014: 174-175) and reveals that there is 
a relationship between information structure and position on the packet, 
as well as that structure is subject to substantial variation. Moreover, 
Sebba concludes that design strategies tend to respond to the needs of 
branding and promotion of a product, just like the inclusion of a number 
of different languages. However, only a limited number of nationally or 
internationally dominant languages appear in product packaging. 

The chapter fills the gap in the studies on the relationship between the 
design of the packaging and the languages incorporated into that design, 
showing a threefold conceptualisation of texts on product packaging: as 
language(s), practices and images. 

The chapters in the volume share an interest in both multilingualism and 
the versatile modalities of communication in the past and now, and some 
also rely on the conceptualisation of “text as image” (cf. the references 
in Peikola and Varila, this volume). The interests and foci of the chapters 
embrace and extend this view by incorporating the dimension of space on 
different levels of language, communicative acts and artefacts (Kopaczyk; 
Sebba; Seiler). This framework (cf. Sebba 2012) enables the making of 
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meaningful connections between language in the social realm and a range 
of specific features in the “grey area between text and image”, further 
graphics (diagrams, tables, cartoons) and peritextual features (changes in 
typeface, use of characters drawn from foreign writing systems) (Peikola 
and Varila; Tyrkkö and Legutko); script, abbreviations and brevigraphs 
(Rogos-Hebda); metalinguistic boundary markers (Włodarczyk and 
Adamczyk); and layout in multilingual glossaries (Seiler). Moving even 
further towards the materiality of multilingual spaces, Mark Sebba 
investigates informational hierarchies in multilingual product packaging 
and views these from the perspective of language policies, underlying the 
attitudes and prestige of different languages. In the realm of social media, 
influencer culture is presented by Limatius as a multimodal and multi- 
lingual social practice, where the use of textual resources and language 
choices is connected to the nature of the hosting platform and its visuality, 
and to aspirational perceptions of self-promotion and language attitudes. 
In addition, chapters in this volume also reflect upon audience design and 
audience guidance in a variety of multilingual texts across space and time. 

In general, the chapters fit the characterisation of multimodality 
studies provided by Bateman et al. (2017), who argue for a “multimodal 
approach that complements any other disciplines we may choose to link 
with” (2017: 24). In our case, the other disciplines constitute linguistics, 
discourse analysis and pragmatics and, in the contributions, the multi- 
modal perspective clearly integrates theories and methods drawn from 
these fields. This is possible as the authors have interpreted extralinguistic 
mode(s) discursively in relation to multilingualism, on top of conven- 
tional, rhetorical or aesthetic deployment of the non-verbal. Another 
common thread is that the authors analyse their data without a specific 
hierarchy of codes in mind, which allows them to observe a spectrum of 
interconnections between the codes, and between the codes and modes. 
In addition, the chapters offer a fresh approach to the nature of the fusion 
of modes conventionally taken for granted in multimodality studies (i.e. 
resource integration in Baldry and Thibault 2006: 18; see Bateman et al. 
2017: 17 for more references). Phenomena in “the grey area between 
text and image”, such as “half-graphic objects” or “visual diamorphs”, 
attest to the permeability of the verbal resources, visuality and page com- 
position, and obtain a special status not only to an analyst, but also to 
the audience. In order to process these, both may take advantage of the 
non-immediacy of written or text-based digital communication to render 
heterogeneous multidimensional grammars of comprehension and mean- 
ingful interpretations. 


1.1.3 Summary 


In this volume, we bring together research spanning different contexts 
of multilingual written communication from the past and the pre- 
sent: from historical manuscripts through to early printed texts and 


Modalities from Medieval to Modern Times 13 


from new media to artefacts of daily life. We are particularly interested 
in the interplay between different languages, channels and forms of 
communication, and in searching for patterns that would allow for 
some generalisations and deepened theoretical reflections on the inter- 
play between different codes and modes that evoke non-verbal associ- 
ations. We think that any generalisations and theoretical advances need 
to be based on an operationalisation of interconnections between the 
use of more than one language (i.e. how the two codes interact), the 
visual mode (i.e. how the modalities interconnect with individual codes 
and, more generally, with the strictly linguistic/written mode), as well 
as the cultural practices involved in social materiality and text trans- 
mission. The volume provides illustrations of the interfaces between 
the multilingual and the multimodal in a (medieval/historical) MS, in 
a Late Modern periodical or in social media content, as well as a pre- 
liminary theorisation and a toolbox needed to bring these together for 
comparison and contrast. 

Languages studied in our volume include medieval Latin and French, 
Old Polish, contemporary Finnish, (Namibian) German, Afrikaans, and 
various periods of English. Illustrations presented by our authors cover 
an even broader range of language sources: (e.g. Scots, Korean, Swedish). 
There is no denying, however, that English features prominently in most 
chapters. The special position of English stems from the fact that the 
resources invested in the documentation and research into both its his- 
torical and contemporary shape have been more robust than in the study 
of other languages. This has resulted in the availability of easily access- 
ible, modern electronic databases, which were not only designed for lin- 
guistic purposes, but have accounted for the multilingual nature of its 
development and usage. Some resources have also incorporated aspects 
of multimodality (e.g. the Punch magazine). Regardless of the language(s) 
discussed, the case studies presented in the volume bring results which 
pave the way for generalisations and extension to cover further data 
representing other languages and periods. 


References 


Baldry, Anthony, and Paul J. Thibault, 2006. Multimodal Transcription and 
Text Analysis: A Multimedia Toolkit and Coursebook with Associated on-line 
Course. Equinox: London and New York. 

Barret, Sébastien, Dominique Stutzmann, and Georg Vogeler, eds. 2016. Ruling 
the Script in the Middle Ages. Formal Aspects of Written Communication 
(Books, Charters and Inscriptions). Tarnhout: Brepols. 

Bateman, John A. 2008. Multimodality and Genre. A Foundation for the 
Systematic Analysis of Multimodal Documents. Basingstoke: Palgrave 
Macmillan. 

Bateman, John A. 2014. Developing a GeM (genre and multimodality) model. In 
Interactions, Images and Texts: A Reader in Multimodality, edited by Sigrid 
Norris and Carmen Daniela Maier, 25-37. Berlin: De Gruyter. 


14 Matylda Włodarczyk, Jukka Tyrkkö and Elżbieta Adamczyk 


Bateman, John A., Janina Wildfeuer, and Tuomo Hiippala. 2017. Multimodality. 
Foundations, Research and Analysis: A Problem-Oriented Introduction. 
Berlin: Mouton. 

Blommaert, Jan. 2010. The Sociolinguistics of Globalization. Cambridge: 
Cambridge University Press. 

Bös, Birte, and Matti Peikola. 2020. Framing Framing: The Multifaceted 
Phenomena of Paratext, Metadiscourse and Framing. In The Dynamics of Text 
and Framing Phenomena in the History of English, edited by Matti Peikola 
and Birte Bos, 3-31. Amsterdam: Benjamins. 

Caroll, Ruth, Matti Peikola, Hanna Salmi, Mari-Liisa Varila, Janne Skaffari, and 
Risto Hiltunen. 2013. Pragmatics on the Page. Visual Text in Late Medieval 
English Books. European Journal of English Studies 17 no. 1: 54-71. 

Casas, Maria C. 2009. Multimodality in Canadian Black Feminist Writing: Orality 
and the Body in the Work of Harris, Philip, Allen, and Brand. Amsterdam & 
New York: Rodopi. 

Claridge, Claudia 2020. Discourse Representation in Early Modern English 
Historiography. In Speech Representation in the History of English: Topics 
and Approaches, edited by Peter J. Grund and Terry Walker, 212-237. 
Oxford: Oxford University Press. 

Fenlon, Jordan. 2015. Seeing Sentence Boundaries: The Production and 
Perception of Visual Markers Signalling Boundaries in Signed Languages. PhD 
thesis, University College London. 

Goffmann, Erving. 1959. The Presentation of Self in Everday Life. Garden City, 
NY: Double-day. 

Grund, Peter J., and Terry Walker 2020. Speech Representation in the History 
of English: Introduction. In Speech Representation in the History of 
English: Topics and Approaches, edited by Peter J. Grund and, Terry Walker, 
1-28. Oxford: Oxford University Press. 

Jaffe, Alexandra. 2000. Introduction: Non-standard orthography and non- 
standard speech. Journal of Sociolinguistics 4 no. 4: 497-513. 

Jewitt, Carey ed. 2017. Handbook of Multimodal Analysis, 2nd ed. London: 
Routledge. 

Kopaczyk, Joanna. 2017. Administrative Multilingualism on the Page in Early 
Modern Poland: In Search of Framework for Written Code-Switching. In 
Multilingual Practices in Language History. English and Beyond, edited by 
Päivi Pahta, Janne Skaffari, and Laura Wright, 275-298. Berlin: Mouton de 
Gruyter. 

Kress, Gunther. 2010. Multimodality: A Social Semiotic Approach to 
Contemporary Communication. London/New York: Routledge. 

Kress, Gunther and Theo van Leeuwen. 1996. Reading Images: The Grammar of 
Visual Design. London: Routledge. 

Kress, Gunther, and Theo van Leeuwen. 2001. Multimodal Discourse: The Modes 
of Media of Contemporary Communication. London: Edward Arnold. 

Leppänen, Sirpa, Samu Kytölä, and Elina Westinen. 2017. Multilingualism and 
Multimodality in Language Use and Literacies in Digital Environments. In 
Language, Education and Technology. Encyclopedia of Language and 
Education, 3rd ed., edited by Steven L. Thorne and Stephen May, 119-130. 
London: Springer. 

Lyons, Agnieszka. 2018. Multimodal Expression in Written Digital Discourse: The 
Case of Kineticons. Journal of Pragmatics 131: 18-29. 


Modalities from Medieval to Modern Times 15 


Maxwell, Kate. 2015. When Here is Now and Now is Then: Bridging the 
Gap in Time with “Sumer is icumen”. In Building Bridges for Multimodal 
Research: International Perspectives on Theories and Practices of Multimodal 
Analysis, edited by Janina Wildfeuer, 359-368. Bern: Peter Lang. 

Maxwell, Kate. 2016. Beyond Sound, Image, and Text: The (More) Hidden 
Modes of the Manuscript. Medieval/ Digital Multimodalities. Seminar for the 
New Chaucer Society2016. Available at http://dhmedievalist.com/scalar/med 
ievaldigitalmultimodalities/medievaldigital-multimodalities. 

Norris, Sigrid. 2004. Analyzing Multimodal Interaction: A Methodological 
Framework. New York: Routledge. 

Pahta, Päivi, Janne Skaffari, and Laura Wright, eds. 2017. Multilingual Practices 
in Language History. English and Beyond. Berlin: Mouton de Gruyter. 

Pearsall, Derek. 1989. “Gower’s Latin in the Confessio Amantis”. In Latin 
and Vernacular: Studies in Late Medieval Texts and Manuscripts, edited 
by Alastair J. Minnis. York Manuscripts Conferences: Proceedings Series 
1. Cambridge: D.S. Brewer, 13-25. 

Peikola, Matti, Aleksi Makilahde, Hanna Salmi, Mari-Liisa Varila, and Janne 
Skaffari, eds. 2017. Verbal and Visual Communication in Early English Texts 
(Utrecht Studies in Medieval Literacy 37). Turnhout: Brepols. 

Peikola, Matti, Jukka Tyrkk6, and Mari-Liisa Varila. 2021. Graphic Literacy 
in the History of English. CofP. ICEHL pre-conference workshop, Leiden, 
August. 

Pennycook, Alastair. 2004. Global Englishes and Transcultural Flows. London: 
Routledge. 

Scollon, Ronald, and Suzie Wong Scollon. 2003. Discourses in Place: Language 
in the Material World. London: Routledge. 

Sebba, Mark. 2012. Researching and Theorising Multilingual Texts. In Language 
Mixing and Code-switching in Writing. Approaches to Mixed-Language 
Written Discourse, edited by Mark Sebba, Shahrzad Mahootian, and Carla 
Jonsson, 1-26. London: Routledge. 

Thomas, Martin. 2014. “Evidence and Circularity in Multimodal Discourse 
Analysis.” Visual Communication 13 no. 2: 163-189. https://doi.org/10.1177/ 
1470357213516725 

van Leeuwen, Theo. 2004. Ten Reasons Why Linguists Should Pay Attention 
to Visual Communication. In Discourse and Technology: Multimodal 
Discourse Analysis, edited by Philip LeVine and Ron Scollon, 7-19. 
Washington: Georgetown University Press. 

Włodarczyk, Matylda, Joanna Kopaczyk, and Elżbieta Adamczyk. 2018. 
Crossing the Language and Discourse Boundaries in the Electronic Repository 
of Greater Poland Oaths, 1386-1444 (ROTHA). Invited presentation at the 
symposium on Understanding Multilingual Sermons of Middle Ages, Vienna, 
Austrian Academy of Sciences. 


